Biometric Social Network

ABSTRACT

A first program component having a access to a body of information comprising biographic an biometric information about people; a second program component adaptable to search the body of information for information linking biometric and biographic information; and a third program component adaptable to identify other people by returning linked information in response to a query of biographic or biometric information entered by a user.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. provisional patentapplication No. 61800374.

BACKGROUND

The subject matter of this specification relates to the field oforganizing and linking biometric and biographical information aboutindividuals.

Information has become one of the most abundant and important resourcesin moderns society. Both private companies and government entities spendbillions of dollars gathering and processing this data. However, oftenthe different sets of information remain in separate databases. Forexample, social networking sites often have information relating to whopeople know, where they travel, what activities they enjoy, etc.Government databases have biographical (address, birth date, etc.) andbiometric (fingerprints, etc.). There is no common linkage between thesesite, even if a user has access to these sites.

In addition, access to this information often requires specific textsearches. Even many search engines that return non-text data such asimages and photographs function by searching text-based meta-tags whichare manually entered. There are instances when instead of text, otherdata such as images are more convenient for queries. For example, youmay see someone at a business conference. It would be advantageous toknow information about them based strictly on their image, particularlyany common links you may have to that person.

SUMMARY

The Biometric Social Network solves this problem by providing a systemthat is able to cross-search between social networking site, publicdocuments on the web, as well as any other database containingbiographical or biometric information about individuals.

The Biometric Social Network is a system that identifies and connectsinformation available on the internet to real-world personalinteraction. In practice, it will provide a means for people to beidentified and associated with other people, places, or commontouchstones through a continually-updated corpus of informationextracted from the internet that contains social, contextual,professional, historic, and other types of information. The system isalso able to cross-search for multiple individuals and identify anylinks between them.

DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a system diagram depicting the core physical machinefunctionality and operating system functionality of a computer.

FIG. 2 is a flow chart depicting the subroutines in one embodiment ofthe Biometric Social Network “program code/system”

FIG. 3 is a system diagram depicting functional components of thebiometric search application.

FIG. 4 is a flow diagram of the biometric segregation subroutine.

FIG. 5 is a flow diagram of the template generation subroutine.

FIG. 6 is a flow diagram of the build database subroutine.

FIG. 7 is a flow diagram of the identification subroutine.

DETAILED DESCRIPTION

Webcrawlers or other information-gatherers comb through the internet(body of available public and non-public information containing imagery,and other information); these webcrawlers or other information gathererscreate a database of identifying imagery and associated informationincluding available metadata. For face information, this may be adatabase of facial templates extracted from images containing images ofpeople, as well as a record or other facial images that appear in commonpictures, metadata such as where or when the image was taken, what siteit appeared on, or in the case of applications such as Facebook orLinked-In, could extract a list of the persons friends or associates.

The information can be accessed multiple ways, including the manualupload of a photograph through a webserver or other portal, or throughthe use of a camera phone or other portable device.

In a simple sub-example, the system can answer the question “who is thisperson in front of me” by taking a picture, querying the corpus ofinternet-derived information, and identifying the individual. To go astep further, in this social example, the system can also answer thequestion, “are the two individuals in front of me acquainted” byidentifying both and a linkage or lack-therof between the two. Thislinkage may be a published image showing both, or extracted and impliedfrom associated data such as both individuals having attended commonevents as determined by archived information, having other individualsthat connect them, or being able to determine that they have previouslybeen in the proximity of one another by residence, or other.

Biographic information are historical facts about a person's life suchas her name, age, weight, occupation, social security or telephonenumber, date of birth, marital status, etc. Biometric information arebiometric indicators of that person, such as fingerprints, dna, handgestures, facial images, gait, iris, etc. Biometric information islinked to biographic information if it references the same person, suchas when a facial image appears on an id card with the person's homeaddress, or when a video of a person walking is referenced by an articlestating their name and describing their occupation.

The system is intended to operate on information in multiple forms; toillustrate by way of an example, we will use facial imagery as aspecific type of available information. In this example, the integratedsystem may operate as follows.

In one embodiment of the invention, a user can capture a photograph ofan individual at a party and submit it as a query into the system asshown in FIG. 7. (alternatively the user could enter certainbiographical information and run the search off that information). Thesystem will then return all information on the internet linking theimage taken to other biometric or biographic information. Optionally,this information can be automatically summarized or returned first to asubsystem, which performs further manipulations before returning resultsto the user such as matching the return to existing databases of knowindividuals.

In one embodiment, the system comprises a computing components selectedfrom the group consisting of a virtual machine and a computer (aphysical machine). Each computer has at least one processor and at leastone storage device.

A processor operable by the computing component executes a first programcode stored in a storage device accessible by the computing componentfor: collecting from the user a first search query which containsidentifying information about a first subject; collecting from the usera second search query which contains identifying information about asecond subject; extracting a first set of biographic or biometricinformation from the first search query; extracting a second set ofbiographic or biometric information from the second search query;searching the body of information using the first set of biographic orbiometric information to produce a first set of search results;searching the body of information using the second set of biographic orbiometric information to produce a second set of search results;comparing the first set of search results with the second set of searchresults to produce a set of common search results; and displaying theset of common search results to the user.

Said search query identifying information may be from the groupconsisting of a digital images, digital video, digital, audio, text,biometric information, or a plurality thereof.

Said biometric information may be from the group consisting offingerprints, dna, hand gestures, facial images, hand writing, gait,iris, voice, or a plurality thereof. Other biometrics are well known toa person of ordinary skill in the art.

Said biographic information are historical facts about a person's life.A non-exclusive list of these facts are a person's name, alias, age,weight, occupation, social security number, identification number,telephone number, date of birth, marital status, or a plurality of.Other biographical information types are well known to a person ofordinary skill in the art.

All or part of the methods described herein may be implemented as acomputer program product that is a non-transitory computer-readablestorage medium encoded with computer code that is executable by aprocessor.

The details of one or more embodiments of the subject matter of thisspecification are set forth in the drawings and descriptions containedherein. Other features, aspects, and advantages of the subject matterwill become apparent from the description, drawings, and claims.

The subject matter of this specification functions in a variety ofcomponent combinations and contemplates all those types of components aperson of ordinary skill in the art would find suitable for functionsperformed. The figures describe specific components in specificembodiments. However the range of the types of components mentioned inthe description of the figures may be applied to other embodiments aswell.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used herein, thesingular forms “a,” “an,” and “the” are intended to include the pluralforms as well, unless the context clearly indicates otherwise. The terms“comprises” and/or “comprising,” when used in this specification,specify the presence of stated features, integers, steps, operations,elements, and/or components, but do not preclude the presence oraddition of one or more other features, integers, steps, operations,elements, components, and/or groups thereof.

The subject matter of this specification is described below withreference to system diagrams, flow diagrams, and screen mockups ofsystems, methods, and computer program products. Except where used inthe claims, the term “system” refers broadly to the subject matter ofthis specification, including embodiments that are, systems, methods, orcomputer program products. Each block or combinations of blocks in thediagrams can be implemented by computer program code and may represent amodule, segment, or portion of code. Program code may be written in anycombination of one or more programming languages, including objectoriented programming languages such as the JAVA®, SMALLTALK®, C++, C#,OBJECTIVE-C® programming languages and conventional proceduralprogramming languages, such as the “C” programming language.

It should be noted that, in some alternative implementations, thefunctions noted in the blocks may occur out of the order noted in thefigures. For example, two blocks shown in succession may, in fact, beexecuted substantially concurrently, or the blocks may sometimes beexecuted in the reverse order, depending upon the functionalityinvolved. It will also be noted that each block or combination of blocksin the diagrams can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts.

Computer program code may be provided to a processor or multipleprocessors of a computer to produce a particular machine, such that theprogram code, which executes via the processor, creates means forimplementing the functions specified in the system diagrams, flowdiagrams, and screen mockups.

The subject matter of this specification is implemented on one or morephysical machines. Each physical machine is a computer comprising one ormore processors and one or more storage devices; however a singleprocessor and a single storage device are sufficient. A person ofordinary skill in the art will recognize the variety of types ofcomputers suitable for the functions described, including desktops,laptops, handset devices, smartphones, tablets, servers, or accessoriesincorporating computers such as watches, glasses, or wearablecomputerized shoes or textiles. A non-exhaustive list of specificexamples of computers includes the following: Dell ALIENWARE™ desktops,Lenovo THINKPAD® laptops, SAMSUNG™ handsets, Google ANDROID™smartphones, Apple IPAD® tablets, IBM BLADECENTER® blade sewers, PEBBLE™wearable computer watches, Google GLASS™ wearable computer glasses, orany other device having one or more processors and one or more storagedevices, and capable of functioning as described in this specification.

A processor may be any device that accepts data as input, processes itaccording to instructions stored in a storage component, and providesresults as output. A person of ordinary skill in the art will recognizedthe variety of types of processors suitable for the functions disclosed,including general purpose processing units and special purposeprocessing units. A non-exhaustive list of specific examples ofprocessors includes the following: Qualcomm SNAPDRAGON™ processors;Nvidia TEGRA® 4 processors; Intel CORE™ i3, i5, and i7 processors; TEXASINSTRUMENTS™ OMAP4430; ARM® Cortex-M3; and AMD OPTERON™ 6300, 4300, and3300 Series processors. Each computer may have a single processor ormultiple processors operatively connected together (e.g. in the“cloud”).

A storage device is any type of non-transitory computer readable storagemedium. A person of ordinary skill in the art will recognized thevariety of types of storage devices suitable for the functionsdisclosed, including any electronic, magnetic, optical, electromagnetic,infrared, or semiconductor system or device, so long as it does notreduce to a transitory or propagating signal. A non-exhaustive list ofspecific examples of storage devices includes the following: portablecomputer diskettes, hard disks, random access memory, read-only memory,erasable programmable read-only memory, flash memory, optical fibers,portable compact disc read-only memory, optical storage devices, andmagnetic storage devices. Each computer may have a single storage deviceor multiple storage devices operatively connected together (e.g. in the“cloud”).

This system may be implemented on one or more computers running one ormore instances of a virtual machine. A virtual machine is a softwareimplementation of a computer that executes programs like a physicalmachine. Thus a single physical machine may function conventionally as aphysical computer, while also implementing a virtual machine that canperform the same processes as the physical computer. Multiple instancesof a virtual machine may run on one computer or across multiplecomputers. A person of ordinary skill in the art will recognize thevariety of types of virtual machines suitable for the functionsdisclosed, including system level virtual machines, process levelvirtual machines, fictive computers, and distributed computers. Anon-exhaustive list of specific examples of virtual machines includesthe following: VMWARE® virtual machines and Oracle VM VIRTUALBOX™virtual machines.

For the purposes of this specification, the term “computing component”means a computer, a virtual machine, or multiple computers or virtualmachines functioning as a single component. The term “computer” islimited to physical machines. Generally a computer functions as acomputing component by implementing an operating system through whichprogram code, which implements the methods of this system, is executed.Generally, when a virtual machine functions as a computing component, acomputer implements a hypervisor which implements a separate operatingsystem, through which the program code is executed.

As referenced above, a single computer may implement multiple computingcomponents, wherein the computer itself functions as a computingcomponent and concurrently implements one or more instances of a virtualmachine. Each virtual machine functions as a separate computingcomponent. Similarly, a plurality of computing components may be made upof separate computers, none of which implement a virtual machine, or aplurality of computing components may be implemented on a singlecomputer wherein only the virtual machines function as computingcomponents. Additional combinations are contemplated as well, such aswhere a computing component is implemented across multiple computers.For example, a hypervisor of a virtual machine may manage the processorsand storage devices of three computers to implement a virtual machinethat functions as a single computing component. A person of ordinaryskill in the art will recognize the range of combinations of computersand virtual machines that are suitable for the functions disclosed.

Each of the plurality of computing components, whether implemented asseparate computers or on a single computer, are operatively connected toone another, such as by a communications network. One skilled in the artwill recognize the appropriate media over which multiple computingcomponents may be operatively connected to each other in a mannersuitable for the functions disclosed, including as a communicationsnetwork that allows the computing components to exchange data such thata process in one computing component is able to exchange informationwith a process in another computing component. The communicationsnetwork may also be a virtual communications network managed by ahypervisor. A non-exhaustive list of specific examples of transmissionmedia includes: serial or parallel bus systems, wireless, wireline,twisted pair, coaxial cable, optical fiber cable, radio frequency,microwave transmission, or any other electromagnetic transmission media.In addition computing components can be operatively connected usingsecure socket layer or HTTPS communications networks employing PKItechniques as described below.

The system allows for the collection of a set of biometric informationfrom a subject. Biometric information is a distinctive, measurable,physiological and behavioral characteristic of an individual. A personof ordinary skill in the art will recognize the range of biometricinformation that can be collected and included in a set of biometricinformation suitable for the functions disclosed. A non-exhaustive listof specific examples of biometric information includes: iris,fingerprint, fingernail, hand, knuckle, palm, vascular, face, retina,deoxyribonucleic acid, odor, earlobe, sweat pore, lips, signature,keystroke, voice, eye vein, hand writing, and gait. A set of biometricinformation may consist entirely of one biometric type or modality, ormultiple types or modalities.

The system collects the set of biometric information through one or morebiometric collectors operatively connected to one or more of theplurality of computers. A person of ordinary skill in the art willrecognize the range of biometric collection devices that are suitable tocollect biometric information, including fingerprint readers, irisscanners, facial recognition imagers, and DNA samplers. A non-exhaustivelist of specific examples of biometric collectors include the Futronic'sFS88 USB 2.0 fingerprint scanner, FBI FIPS 201 compliant fingerprintscanners, AOPTIX STRATUS™ iris scanners, FBI FIPS compliant irisscanners, the BI2 MORIS™ facial recognition device, the Bode TechnologyBUCCAL DNA COLLECTOR™, L-1 Identity Solution's HIIDE™ device, SecurePlanet's BRAVE™ system, SRI International's IRIS ON THE MOVE® systems,and Bayometric Inc.'s voice authentication system.

Records of biometric information associated individuals are stored asbiometric application databases in one or more storage devices.Databases are organized collections of data and include softwareapplications that allow for the definition, creation, querying, update,and administration of the organized collections of data. A person ofordinary skill in the art will recognize the range of types of databasessuitable for functions disclosed, including active databases, clouddatabases, distributed databases, federated database systems, andunstructured database systems. A non-exhaustive list of specificexamples of databases includes: MySQL, PostgreSQL, SQLite, MICROSOFT®SQL Server, Microsoft Access, Oracle, SAP, and IBM DB2.

The above components are described in greater detail below withreference to the figures. The descriptions below set forth the variousprocesses, relationships, and physical components of various embodimentsof the subject matter of this specification.

FIG. 1 is a system diagram depicting core physical functionality andoperating system functionality of a computer. Computer hardware 103consists of a processor(s) 105, display device(s) 107, input device(s)109, network device(s) 111, and storage device(s) 113. The operatingsystem software 115 manages computer hardware resources and dictates theexecution of all other software programs and processes. The operatingsystem additionally controls the user interface 117, file system andmemory management 119, access control 121, user applications 123, andnetwork interface 125 of a functioning computer. The operating systemcan be multi-user, multiprocessing, multitasking, multithreading,real-time, and the like. The operating system performs basic tasks,including but not limited to: recognizing input from input device(s);sending output to display device(s); and keeping track of files anddirectories on storage device(s). The operating system includes variouscomponents for establishing and maintaining network connections (e.g.,software for implementing communication protocols, such as TCP/IP, HTTP,Ethernet, USB, FireWire® protocols, etc.).

FIG. 2 is a flowchart depicting the subroutines in one embodiment of theBiometric Social Network “program code/system,” wherein linkages betweenpersons are detected. A processor 105 operable by the computingcomponent 101 executes a first program code stored in a storage device113 accessible by the computing component. A graphical user interface isshown on the display 107. With the input device 109, the user inputsknown data 1705 about the first subject. With the input device 109, theuser then inputs known data 1707 about the second subject. In oneembodiments, the data on both subjects may be entered concurrently. Saiddata may be biographical text information, digital media, biometricinformation, or biometric template information. Said data may alsocontain information about a person or persons other than subject 1, forexample, a photograph containing subject 1 and one of subject 1's knownassociates.

A first set of biographical search terms and biometric templatesmetadata 1710 are extracted from the first subject's associated data1705. The first set of metadata 1710 is submitted 1715 to a first searchengine which searches a first database. In one embodiment, a searchengine searches a plurality of databases. In an alternate embodiment,the metadata 1710 may be submitted to a plurality of search engines. Ina parallel set of processes, a second set of biometric and biographicalmetadata will be extracted 1712 and submitted 1717 to the first searchengine. In one embodiment, processes 1707, 1712, and 1717 may beexecuted after processes 1705, 1710, and 1715.

The first set of search results pertaining to the first set of metadataand the second set of search results pertaining to the second set ofmetadata are compared 1725 based on methods known to those of ordinaryskill in the arts. In one embodiment, the intersection set of the firstset of results and of the second set of results is calculated. Adecision 1730 is made depending on the comparison. If any similaritiesare found 1735 (the intersection set is not empty), the intersection setis displayed on the users display device 107 and stored on the user'sstorage device 113. In an alternative embodiment, the first set ofresults and second set of results are stored on the user's storagedevice 113. If no similarities are found 1740 (the intersection set isempty), a message indicating that no links was found is displayed on theusers display device 107.

It is understood by those of ordinary skill in the arts that portions ofthe processes may be run on a second or plurality of computers.

FIG. 3 is a system diagram depicting an embodiment showing functionalcomponents of the biometric search application 201. When embodiments ofthe invention run searches of biometric information, the subroutinedepicted in FIG. 3 is one example of a component that executes thesearch. In this embodiment the biometric search application executesover a plurality of computing components. The biometric searchapplication contains an operator user interface 211 that provides aninterface for collecting biometric data, performing biometric searches,validating and managing PKI certificates, and configuring local andremote biometric application databases. The operator user interface alsomanages biometric template extraction using the biometric templateextractor 217 and provides an interface to all supported biometriccollectors 221.

FIG. 4 depicts the subroutine that is executed when multiple modalitiesof biometric information is collected. A simple collection would consistof collecting only one biometric modality from a subject. A segregatedcollection would consist of collecting a plurality of biometricmodalities. FIG. 4 demonstrates a plurality of biometric modalitiesbeing collected by an accompanying biometric collector. The appropriatebiometric collector must be operatively connected to a computingcomponent and interface with the biometric search application. All datacollected by the biometric collector is accepted as raw biometric data.The data received by a biometric collector may be associated with aparticular data field, which will generally correspond to a specificbiometric modality or a specific physical parameter of the biometricdata collected (“type”). Data fields may be configured as needed by thenature of the biometric information required to be searched against. Forexample a biometric collector may collect a full set of ten-prints andhave separate data fields for each finger collected as well as the setas a whole, or an imaging device may collect facial and iris informationin a single collection but associate iris data as one data field and thefacial data as a second data field. Based on these data fields, the typeand modality of the biometric collected is determined.

After the data is collected, it is then segregated into subsets ofbiometric information associated with each biometric applicationdatabase of the system. For example, there may be three entitiescontrolling biometric application databases in the system. The firstentity may have records associated with fingerprint biometrics, thesecond with facial biometrics, and the third with iris. In anotherexample, all three participating organizations may control biometricapplication databases associated with fingerprint biometrics; howeverthe first may only contain thumbprint data, while the other two containfull ten-print sets. In either example the original set of biometricinformation would be segregated into subsets consistent with the typeand modality of biometric information associated with the biometricapplication databases of each participating organization. Aftersegregation, the biometric information is sent to be generated into abiometric template using the appropriate template generation algorithmfor that particular biometric type or modality.

FIG. 5 is a flow diagram depicting a template creation subroutineexecuted by biometric template extractor. Raw biometric data such as aten digit set of fingerprints is collected using a biometric collector.The minutiae that enable fingerprint identification are then extractedfrom the raw biometric data. The template creation algorithm associatedwith the modality to be searched against is selected and applied to theextracted features to generate the applicable searchable template forthe type or modality. Software implementing biometric templategeneration algorithms are well known in the art, including the CROSSMATCH® fingerprint template generator and Neurotechnology MEGAMATCHER™fingerprint template generator.

FIG. 6 depicts a flow diagram for the creation of a database of knownindividuals for reference purposes. The process collects a set ofimages, extracts identifiable biometric features from those images,identifies the personnel based on pre-existing biometric records ofpersonnel, such as driver's licenses or social media profiles, andcompares newly collected images to find identifying biometrics andcontinue to build and update the database.

FIG. 7 depicts the basic subroutine for determining identifyinginformation for a given individual whose biometric information iscollected with an embodiment of the invention.

The description of the subject matter of this specification has beenpresented for purposes of illustration and description, and is notintended to be exhaustive or limited to the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art. The embodiments were chosen and described in order to bestexplain the principles of the system, practical applications, and toenable others of ordinary skill in the art to understand variousembodiments with various modifications as are suited to the particularuse contemplated.

1. A system for providing social networking between users subscribing tosaid system from communication devices, said system comprising: a firstprogram component having a access to a body of information comprisingbiographic a biometric information about people and operativelyconnected to a biometric collector for capturing additional biometricinformation; a second program component adaptable to search the body ofinformation for information linking biometric and biographicinformation; and a third program component adaptable to identify otherpeople by returning linked information in response to a query ofbiographic or biometric information entered by a user.